Flange Integrity Classification Using Artificial Intelligence

ABSTRACT

A computer-implemented method for flange integrity classification using artificial intelligence is described. The method includes obtaining images of a flange, wherein an image of the images is captured at a predetermined angle of image capture. The method includes classifying a condition of the flange using a trained machine learning model. Further, the method includes rendering an indication of the condition of the flange.

TECHNICAL FIELD

This disclosure relates generally to machine-learning based flangeintegrity classification.

BACKGROUND

Flanges include projecting collars that are physically coupled to seal apressurized vessel or pipe. Multiple flanges are used in a pipeline.Standards applicable to flanges are promulgated by organizations such asthe American Society of Mechanical Engineers (ASME) and AmericanNational Standards Institute (ANSI).

SUMMARY

An embodiment described herein provides a method for flange integrityclassification using artificial intelligence. The method includesobtaining images of a flange, wherein the images are captured at apredetermined angle of image capture. The method includes classifying acondition of the flange using a trained machine learning model. Themethod also includes rendering an indication of the condition of theflange.

An embodiment described herein provides an apparatus comprising anon-transitory, computer readable, storage medium that storesinstructions that, when executed by at least one processor, cause the atleast one processor to perform operations. The operations includeobtaining images of a flange, wherein the images are captured at apredetermined angle of image capture. The operations include classifyinga condition of the flange using a trained machine learning model. Theoperations also include rendering an indication of the condition of theflange.

An embodiment described herein provides a system. The system comprisesone or more memory modules and one or more hardware processorscommunicably coupled to the one or more memory modules. The one or morehardware processors is configured to execute instructions stored on theone or more memory models to perform operations. The operations includeobtaining images of a flange, wherein the images are captured at apredetermined angle of image capture. The operations include classifyinga condition of the flange using a trained machine learning model,wherein a machine learning model is trained to classify the condition ofthe flange using synthetic images. Additionally, the operations includerendering an indication of the condition of the flange.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an illustration of coupled flanges.

FIG. 2 shows a series of screenshots from a mobile application executingon an industrial tablet.

FIG. 3 is a block diagram of training a machine learning model forflange integrity classifications using artificial intelligence.

FIG. 4 is a block diagram of training a machine learning model forflange integrity classification in a two-part network.

FIG. 5 is an illustration of synthetic flange images.

FIG. 6 is a process flow diagram of a process for flange integrityclassification using a trained machine learning model.

FIG. 7 is a schematic illustration of an example controller (or controlsystem) for flange integrity classification using artificialintelligence according to the present disclosure.

DETAILED DESCRIPTION

Flanges enable connections between pipes of pipeline systems thattransport various materials, such as oil, gas, water, and the like.Additionally, flanges can securely seal pressurized vessels that storevarious materials. A flange can include two flange collars boltedtogether using a set of bolts mated with corresponding nuts. A seal iscreated between the flange collars, enabling a strong joint orconnection between two pipes. Ensuring the integrity of flanges is vitalto maintaining a safe pipeline or pressurized vessel. In some cases,inspections of flanges in the field are conducted infrequently, sincetraditional inspections require that a trained operator travel to thepipeline and visually inspect the flanges. Additionally, once on-site,in traditional inspections the operator may find that flanges arelocated in difficult to view or hidden locations due to the design ofthe pipeline. Further, inexperienced operators are often unable toproperly evaluate conditions of the flanges in traditional inspections.Accordingly, in traditional techniques operators require extensive,costly, and time consuming training in order to evaluate flanges.

Embodiments described herein enable flange integrity classificationusing artificial intelligence. In particular, the present techniques useenable an automated assessment of the integrity and health of one ormore flanges. Using an industrial tablet pre-installed with a mobileapplication, a systematic and automated procedure to assess theintegrity and health of flanges is provided. In examples, theapplication prompts an operator to visit specific flanges and captureimages from specified angles. The application, using these images, willdetermine if any conditions are present using a trained machine learningmodel.

Traditional flange integrity classification is limited to detection ofloosened bolts by analyzing a change in an angle of the bolt over timeor acoustic inspection techniques. Other traditional techniques usePhase Array Ultrasonic Testing (PHAT), where a probe is focused andelectronically swept to scan an area of interest, without moving theprobe. The use of the probe consumes a relatively long period of timefor inspection, and a trained operator is required to operate the probeand interpret the results. As a result, traditional techniques arelimited to testing for loosened bolts or using ultrasonic testing. Thepresent techniques detect a variety of flange conditions that can causefuture leaks, and do not require a trained operator. Image capture isperformed to obtain images of a flange from multiple angles. Forexample, an untrained person (e.g., without specialized knowledge)captures images of flanges from multiple angles as instructed by themobile application. In some embodiments, the mobile application can usedfor training of operators. In examples, an electronic device (e.g., arobot, drone) captures images of flanges from multiple angles asinstructed by the mobile application. A trained machine learning modeltakes the captured images as input, and outputs a predicted condition ofthe flanges.

FIG. 1 is an illustration of coupled flanges 100. In some embodiments,the coupled flanges 100 are included in a pipeline. For example, thepipeline system can be a gathering system, transmission system, or anddistribution system with multiple flanges forming joints or connectionsacross pipes of the pipeline. The flanges are often under high pressureand can transport hazardous or flammable material. Accordingly, flangesin poor condition can create a dangerous environment.

In the example of FIG. 1 , the coupled flanges 100 include a firstcoupled flange pair 110, a second coupled flange pair 130, and a thirdcoupled flange pair 150. The first coupled flange pair 110 includes anumber of bolts 112, 114, 116, and 118. The bolts extend through boltholes in collars 120A and 120B of the coupled flange pair 110. A numberof nuts 122, 124, 126, and 128 receive the threaded end of the bolts112, 114, 116, and 118. Tightening the nuts 122, 124, 126, and 128secures the collars 120A and 120B of the coupled flange pair 110,creating a seal between pipe 101 and pipe 102.

In the example of FIG. 1 , the second coupled flange pair 130 includes anumber of bolts 132, 136, and 138. The bolts extend through bolt holesin collars 140A and 140B of the coupled flange pair 130. A number ofnuts 142, 146, and 148 receive the threaded end of the bolts 132, 136,and 138. The nuts 142, 146, and 148 are threaded onto respective bolts132, 136, and 138.

The coupled flange pair 130 exhibits a number of conditions. Inexamples, a condition refers to a state of a flange with regard to itsappearance, quality, or working order. In examples, the condition is anormal, healthy condition wherein the flange is configured according topredetermined maintenance instructions and bolt tightening data (e.g.,bolting patterns, torque and tensioning figures, procedures, techniquesand recommended controlled bolting equipment). In some embodiments, anormal condition is determined based on a flange type and standardspromulgated by organizations such as the American Society of MechanicalEngineers (ASME) and American National Standards Institute (ANSI). Insome embodiments, the condition is poor or unhealthy, where the flangeexhibits defects that reduces the integrity of the coupled flange. Inexamples, defects include short bolting, missing bolts, angular orparallel misalignment of the flange heads, missing screws, and the like.In some embodiments, a condition of the flange is critical. A flange incritical condition exhibits a severe flange abnormality that can causeimmediate disruption to a system that includes the flange.

A short bolting defect is illustrated using bolt 132 and nut 142. Theshort bolting defect is indicated by the bolt 132 being short andfailing to extend fully through bolt holes of the collar 140A and thecollar 140B. At bolt holes 134 and 144, a missing bolt defect isillustrated. A missing bolt is indicated by bolt holes without bolts,and reduces the integrity of the coupled flanges. The collars 140A and140B of the coupled flange pair 130 are misaligned as shown at referencenumber 131. In particular, the collar 140A and collar 140B show aparallel misalignment, which is indicated by an offset 131 between acenterline 135 of the collar 140A and a centerline 133 of the collar140B.

The third coupled flange pair 150 includes a number of bolts 152, 154,156, and 158. The bolts extend through collars 160A and 160B of thethird coupled flange pair 150. A number of nuts 162, 164, 166, and 168receive the threaded end of the bolts 152, 154, 156, and 158. The nuts162, 164, 166, and 168 are threaded onto respective bolts 152, 154, 156,and 158. The collars 160A and 160B of the coupled flange pair 150 aremisaligned as shown at reference number 151. In particular, the collar160A and collar 160B show an angular misalignment, which is indicated byan angle 151 between a centerline 155 of the collar 160A and acenterline 153 of the collar 160B.

Other defects of a flange include, for example, scratches, gouges, pits,and dents. In examples, scratches are caused by contact with hard,abrasive materials, and can result from mishandling in transit or fromthe removal of protective coatings. In examples, gouges are created by adull object dragging across the flange face, such as a screwdriver,flange. Gouges can be in transit from the fabrication plant to site, orduring commissioning. Pits are small rounded areas of material loss,sometimes in groups and caused by corrosion. In examples, pits arecreated after the flanges are operational for a period of time.Similarly, dents can be caused during the installation and commissioningphases through impact with equipment such as cables, rigging andpositioning of mating flanges.

For ease of illustration, the defects in FIG. 1 are shown as visuallyobserved by the human eye. However, in some embodiments, defectsassociated with the coupled flanges are small and slight as to not bevisible to the human eye. Additionally, a single flange or a flange paircan exhibit any number of defects. For example, a flange pair canexhibit missing bolts, short bolts, parallel misalignment, angularmisalignment, scratches, gouges, pits, dents, and loose bolts eitheralone or in various combinations.

The present techniques include an intelligent system that detects faultyflanges that could eventually leak. Flanges that are not tightened well,have loosened bolts, missing bolts, or short bolting do not retainsufficient force to prevent inner liquid from leaking through theflanges, and are more susceptible to leaks. Moreover, flanges thatfeature angular or parallel misalignment have higher probability toexhibit leaks than perfectly aligned flanges. Hence, detecting thesefaults or anomalies in flanges as early as possible reduces thelikelihood of leak incidents as well as maintenance costs associatedwith damaged flanges.

For ease of description, the flanges described herein are describedgenerally with flange collars and a number of bolts. However, theflanges can be of many different types, such as Weld Neck Raised Face(WNRF), Socket Weld (SW), Slip-On Flange, Flat Faced (FF), Lap Joint,Ring Joint, Threaded Flange, Reducing Flange, Blind Flange, and thelike. Face types of the flanges include, for example, flat face, raisedface, ring joint face, tongue and groove, and male and female faces.Additionally, in examples, the flanges can include a number of finishes,such as serrated or smooth. Flange integrity classification of varioustypes of flanges is enabled by the present techniques, as the basis ofanomaly / defect identification is image analytics augmented withmachine learning capabilities.

FIG. 2 shows a series of mobile application screenshots via displays202A, 202B, and 202C (collectively referred to as displays 202) renderedat respective industrial tablets 200A, 200B, and 200C (collectivelyreferred to as industrial tablets 200). For ease of description, themobile application is described as executing on the industrial tablets200. However, the mobile application according to the present techniquescan be executed on any electronic device, such as a tablet, laptop,desktop, commercial tablet, smartphone and the like. In someembodiments, the industrial tablets 200 include built in sensors thatdetect gas leaks, hazardous or flammable plumes, and the like. For easeof illustration, particular mobile application screenshots areillustrated. However, the mobile application according to the presenttechniques can have any number of screenshots corresponding to renderedprompts, information, and instructions described herein. Further, themobile application is not limited to the visual display of prompts,information, and instructions. In examples, the mobile application cangenerate prompts, information, and instructions using output devicesincluding but not limited to speakers, remote displays, light indicators(e.g., traffic lights). Further, the mobile application can outputhaptic feedback via the industrial tablet, such as vibrations, to guidean operator.

In some embodiments, the mobile application has the ability to measure awide multitude of flange sizes in different locations and conditions.The mobile application can detect a flange in an image and, if present,defects with their respective locations on an image. In examples, afirst industrial tablet 200A includes a display 202A. A prompt toinstruct a user to capture images of a flange is rendered on the display202A. In some embodiments, the prompt includes instructions for imagecapture of a flange from predetermined angles or positions. In someembodiments, the predetermined angle of image capture refers to enablingpanoramic views of a flange joint connection to ascertain maintenancejob precision enabling the application to evaluate the flange joint andidentify defects, if any. For example, identification of parallel /planar misalignment uses predetermined angles of image capture relativeto the flange that captures a front view, side view and top view of theflange.

In FIG. 2 , a second industrial tablet 200B includes a display 202B. Amap 204 is rendered on the display 202B. The map 204 shows the flangesthat operator needs to capture and the flanges that have already beencaptured for inspection. In the example of FIG. 2 , locations 206, 208,and 210 that have been previously visited by the operator are markedusing an item rendered on top of the map 204. In some embodiments, themap 204 is a real world image of the system, with augmented realityelements superimposed on top of the image. In some embodiments, the map204 is a digital map.

Step by step directions are provided to guide the operator towards anext flange location, positioned at location 212. The operator capturesone or more images of the flange using an image capture mechanism (e.g.,camera or camera sensor) of the industrial tablet. In the example ofFIG. 2 , the operator is guided to a next Flange A at Location X. Theuser can visit Flange A at Location X, and capture images of Flange A atAngle 1 and Angle 2 as prompted via display 202A. In some embodiments,on-screen instructions are rendered for training purposes. For example,an operator in training is guided by instructions generated by themobile application. In some embodiments, the mobile application promptsthe operator for further advanced capturing to acquire supplementaryimages of the flange. For example, if a defect is detected, the operatoris prompted to capture additional images. This will facilitate a fastertroubleshooting process in the means of abnormality severity leveldetection. In some embodiments, the operator is prompted to takeadditional images when a first set of captured images is of poorquality, blurry, dimly lit, or unfocused.

In some embodiments, the industrial tablet includes a GPS sensor thatuses GPS a location of the industrial tablet to determine the particularflange being captured. Location identification of flanges is enabledthrough a Geographic Information System (GIS) feature of assets (e.g.,linear assets, in particular considering pipeline is a linear asset).GIS / Linear Reference System data is strengthened with coordinates(latitude, longitude) to enable accurate tracing and mapping of thebreak flanges, valves, associated pipe fittings etc. to enable expeditedservices in operation and maintenance. Reversibly, since the location ofthe each flange is known, the industrial tablet could determine its GPSlocation using the embedded GPS sensor and subsequently determine atwhich flange the tablet is located. This can be used to immediatelyidentify which flange in the system the operator is capturing. In someembodiments, the industrial tablet includes a scan functionality to scana quick response (QR) code permanently attached to the flange, whichincludes the flange’s unique identification number. QR code tags can bealso equipped with radio frequency identification (RFID) tags forquicker identification of the flanges.

A third industrial tablet 200C includes a display 202C. An image of theflange is rendered with augmented reality elements superimposed atop ofthe flange A. As illustrated, nuts, washers, and bolts are shown witharrows as augmented reality elements 216. The present techniques useaugmented reality to superimpose, in real time, detected defects on topof an image of the flange, along with other information. In examples,the other information is details about the flange j oint which includesflange size and rating (e.g. size in inch diameter with pressure ratingin pounds; gasket type, flange type, process fluid in the subject alongwith characteristics like pressure /flow / temperature etc.). Thismobile application can also be connected in real-time with DistributedControl System (DCS) and Document Management System (DMS), etc. toenable accurate and quick data capturing for the subject flange jointunder consideration. In some embodiments, the augmented reality elementsare an indication of the condition of the flange. In examples, a shortbolting defect is rendered using augmented reality elements showing ashort bolt that fails to extend fully through bolt holes of the flangecollars. A missing bolt defect is rendered using augmented realityelements showing bolt holes without bolts, and a parallel misalignmentis rendered using augmented reality elements showing an offset between acenterlines of the flange collars. Similarly, an angular misalignment isrendered using augmented reality elements by showing an angle between acenterlines of the flange collars.

In some embodiments, an indication of the condition of the flange isrendered at industrial tablets 200 using message that provides theoperator with the type defect. For example, the text can be one or moreof “short bolt,” “missing bolt,” “loose bolt,” “parallel misalignment,”or “angular misalignment.” In some embodiments, instructions to remedythe defect are provided to the operator. The industrial tablet renders awarning to operators when a severe flange abnormality is detected (e.g.,missing screw or bolt, significant flange misalignment, missing gasketetc.)

To determine conditions of the flange, cameras of the industrial tablet(e.g., industrial tablets 200) capture images of the flange and apply adeep leaning algorithm that classifies flanges based on their physicalarrangement. The defects occur externally with respect to the flange,and can be detected visually using a vision sensor (e.g., camera). Insome embodiments, a recursive neural network deep learning algorithm istrained using properly labeled images of flanges in healthy and poorcondition, and the machine learning model is trained to classifyflanges. In examples, the machine learning models according to thepresent techniques are integrated on industrial tablets to supportmaintenance personnel (e.g., operators) in performing flange surveys anddetecting anomalies that are intuitively visible. The maintenancepersonnel will direct the tablet’s camera to flanges while the survey isperformed, and the mobile application determines if the physicalarrangement of the captured flanges meets predetermined specifications.In some embodiments, required specifications of the flanges aredetermined from a database of a Document Management System (DMS) of thesubject operating facility. For example, a DMS of an operating facilityis located on a centralized server. A clone of the database can becopied to a cloud server where other edge devices (tablets) cancommunicate. Additionally, partial copies of the database can also bestored locally on the tablet in case server is not reachable. The devicehosting the mobile application is communicatively coupled to the DMS inonline (Industrial WiFi) / offline mode to fetch the technicalspecifications.

The mobile application includes machine learning models trained toclassify flange integrity. The machine learning model takes as input oneor more images of flanges and outputs a classification of flangeintegrity. The classified flange integrity identifies conditions of theflange, including defects present at the flange. In some embodiments,the machine learning model is trained to classify flange integrity usingimages of the flanges as input, as described with respect to FIG. 3 . Insome embodiments, the machine learning model is trained to classifyflange integrity by segmenting images of flanges and inputting the imagesegments to train the machine learning model, as described with respectto FIG. 4 .

FIG. 3 is a block diagram of training a machine learning model forflange integrity classifications using artificial intelligence. Imagesof flanges with known defects are captured. The known defects are usedto labels the images, and the labeled images are used to train themachine learning models at block 304. In some embodiments, the machinelearning models are trained using synthetic images as described withrespect to FIG. 5 . The trained machine learning models output flangeintegrity classifications as indicated at block 306. In examples, theflange integrity classifications include defects/abnormalities of theflange and a bounding box associated with each respectivedefect/abnormality. In some embodiments, the bounding box corresponds toa real-world location associated with the classified defect orabnormality. In some embodiments, the mobile application uses theclassified defect and associated bounding box to superimpose augmentedreality elements indicated by the defect onto an image of the capturedflange in real time, as described with respect to FIG. 2 .

FIG. 4 is a block diagram of training a machine learning model forflange integrity classification in a two-part network. In the example ofFIG. 4 , the two-part network includes image segmentation and training amachine learning model to classify flanges. In some embodiments, imagesegmentation is performed using a convolutional neural networkarchitecture for fast and precise segmentation of the images, and objectdetection is performed using real time object detection. For example,image segmentation is structured similar to a U-Net model, and theobject detection of the abnormalities is modeled using a You Only LookOnce (YOLO) network.

Images of flanges are captured, as illustrated at block 402. In someembodiments, the machine learning models are trained using syntheticimages as described with respect to FIG. 5 . At block 404, the imagesare segmented. For example, for each flange image, the mobileapplication will first dissect and segment each pixel to a flange’scomponent, such as flange body, screw, and bolt. At block 406, thesegmented images are illustrated. In some embodiments, the segmentedimages include sub-images that correspond to the flange body, screw, andbolt. The segmented images are labeled using the condition of theflange-sub image. For example, bolts and screws are labeled according toconditions, such as being tight, loose, or missing. The flange body islabeled according to conditions, such as faces being aligned ormisaligned.

The segmented images are input to machine learning models 408 fortraining. A classifier is trained used to detect and locateabnormalities within the flange using the segmented images. The trainedmachine learning models output flange integrity classifications. Inexamples, the classification includes defects/abnormalities of theflange and a bounding box associated with the defects/abnormalities. Insome embodiments, the bounding box corresponds to a real-world locationassociated with the classified condition, including any defect orabnormality. In some embodiments, the mobile application uses theclassified condition, including the defect or abnormality, andassociated bounding box to superimpose augmented reality elements ontoan image of the captured flange in real time, as described with respectto FIG. 2 .

In examples, the machine learning models described with respect to FIGS.3 and 4 are deep neural networks that require large labeled datasets inorder to be properly trained to recognize patterns and generalize to newimages not encountered previously. Images of flanges with labeleddefects are scarce. Moreover, the images of flanges are typicallymanually labeled, which can be time consuming and tedious. FIG. 5 is anillustration of synthetic flanges 500. The synthetic flanges aregenerated using a sim-2-real transfer in machine learning. For example,simulations are used to generate synthetic flange images for the purposeof training machine learning models. The simulated data represents awide variety of healthy and unhealthy flanges, and is representative ofthe real-world data. Machine learning models are trained by using thesynthetic images, and the flange integrity classifications generated bymachine learning models trained using synthetic images generateclassifications of flange integrity using real-world images.

Simulated data is used to avoid the aforementioned manual labeling anddue to the lack of real images of faulty flanges. Machine learningmodels typically require datasets with at least 5,000 images (andsometimes reach the millions). The simulated data is generated usingthree-dimensional (3D) modeling software (e.g., Blender). Manyparameters of the simulated environment and gauge are randomly sampledin both the faulty and normal state to create a sufficiently largedataset to use in training the machine learning model.

In some embodiments, the machine learning models are trained and thenconverted to reduce a size of the trained machine learning models.Tablets typically use different architectures than devices used to trainmachine learning models. In some cases, tablets are computationallyweaker than devices used to train machine learning models. Therefore, inorder to execute the trained machine learning model on a tablet, it isconverted into a format compatible with the tablet’s architecture. Theseformats use datatypes that hold limited information. Moreover, someoptimizations and trimming procedures are applied to the trained machinelearning model to further reduce its size and inference time (e.g., timeto process the image and provide an output). In examples, a machinelearning library is used for training and inference of deep neuralnetworks, while a lite version of the machine learning library executesthe trained deep neural networks. The lite version of the machinelearning library is executed on mobile or embedded devices. In someexamples, the mobile or embedded devices have limited compute, memory,and power resources. In some embodiments, the Tensorflow is a libraryused in regular deep learning models on relatively complex devices whileTensorflow Lite is used for devices that have limited resources in termsof computation and memory.

In examples, the machine learning models are trained and then quantizedto reduce a size of the trained machine learning models. In someembodiments, quantization constrains weights of the trained machinelearning model to obtain a lightweight trained machine learning modelthat is operable using fewer compute resources when compared to thecompute resources used to train the machine learning model. The mobileapplication is equipped with a dataset of all critical flanges in agiven area, along with the computationally light version of the machinelearning model for classification on the tablet. Accordingly, in someembodiments the lightweight trained machine learning model is deployedat an industrial tablet, such as industrial tablets 200 of FIG. 2 . Insome embodiments, the industrial tablet stores recorded data capturedlocally and transmits it to a server whenever a connection (e.g., datatransmission on a network) is available.

The trained machine learning model classifies images of flanges capturedby the industrial tablet. In some embodiments, the images of flangescaptured by the industrial tablet are processed by the mobileapplication. In some embodiments, the mobile application identifies thetype of flange. The flange can be identified based on a neck type,flange dimensions, etc. In some embodiments, the mobile applicationdetermines the specifications of the flange. Specifications of theflange include, for example, the number of bolts, neck spacing andvertical and horizontal misalignment. In some embodiments, thespecifications are known and obtained from ASME 16.5 standards.

In some embodiments, the captured images are input to the trainedmachine learning models and the classified flange integrity output bythe trained machine learning models is uploaded to a remote server or adatabase for further analysis and management. The database includeshistorical records of flange conditions and updated images of flangeswith defects that can be reviewed and confirmed remotely by maintenanceexperts. In some embodiments, once faulty flanges are confirmed, anotification is transmitted to a corrective maintenance group toschedule maintenance jobs. In examples, the notification includes anidentity, location, and determined condition (including any defects) ofthe flange.

In some embodiments, the images of flanges are captured for input totrained machine learning models located on a remote server. Theindustrial tablet can transmit the captured images wirelessly to thetrained machine learning models installed on the server for analysis ofthe data collected. The tablet stores the images sequentially. In someembodiments, the images are transmitted from the tablet to a serverwhere they are stored sequentially or in time series data and used fordata analytics and modeling. Sequential or time-series data refers toreadings that are stamped with the time of reading. For example, flangeA had minor misalignment within acceptable tolerance on October 28, andthe same flange had an even greater misalignment on November 25. Storingdata in this fashion enables changes overtime to be monitored.Additionally, future conditions are predicted using the sequential datato determine if an intervention is needed.

In examples, in data analytics the server collects time-seriesstatistical information about the health status of all the flanges inthe plant. A server manager analyzes the flange defects and createscorrelation maps between a defective flange and other defective flangesin the plant. In some embodiments, correlation relationships are createdfor defective flanges and other alarms/flags in the process. Forexample, a misaligned flange was detected in a plant, and after a coupleof weeks pump in the same line as the misaligned flange was defected.Both incidents can be correlated for causality. In examples, duringmodeling flange images are used to enhance and improve a flange failuredetection algorithm model. Using the analytics output, a predictionmodel is developed to predict future failures of flanges. In examples,the predicted future failures are used to schedule preventivemaintenance.

In some embodiments, the flange data stored in the cloud system isaccessed for further analysis, and also to check the history of theassets integrity. For example, personnel in a control room (e.g.,central space where a large physical facility or physically dispersedservice can be monitored and controlled) can access the flange data forfurther analysis. In some embodiments, a communication hub isestablished between an operator on location at the plant, and personnelin a remote control room. This will integrate multiple work-relatednecessary tools in one. In examples, the communication hub includes aserver collecting data from different platforms and systems, andpublishing results to a dashboard that can be accessed by operators andengineers. The communication hub connects operators in a centrallocation and maintenance / operator craft on field in real-timeutilizing digital capability like augmented reality and mobility toenhance two way interaction between different functional units of theoperating facility. The communication hub enables integration ofmultiple user functionalities like work permits, equipment data sheets,real time operating parameters, piping and instrumentation diagram(P&ID) / 3D models of plant, minimum maintenance requirements, jobsafety analysis (JSA) etc. In some embodiments, a backend analyzersystem flags flanges that exhibit one or more defects, and sends arequest to the maintenance crew to replace or correct the defectiveflange. The status of the defective flange is updated by the maintenanceteam once the flange is corrected. The communication hub links a flangeclustering system and maintenance system together and display the datain one platform.

FIG. 6 is a process flow diagram of a process for flange integrityclassification using a trained machine learning model. In someembodiments, the machine learning models are trained as described withrespect to FIGS. 3 and 4 . The present techniques introduce a systematicand automated procedure to assess the integrity of flanges. Anintrinsically safe tablet (e.g., tablets 200 of FIG. 2 ) pre-installedwith inspection software (e.g., mobile application) prompts operators tovisit a specific flange and obtain multiple images of the flange, andthe flange integrity is classified. In examples, an intrinsically safetablet is a device that can be used in industrial facility classified ashazardous. Hazardous areas, such as hydrocarbon facilities, can requireany electrical device operated in constrained areas to be sealed in away such that it cannot produce any sparks that could cause ignitions.

At block 602, images of a flange are obtained. An image of the flange iscaptured at a predetermined angle of image capture. In some embodiments,an industrial tablet (e.g., industrial tablet 202) includes one or moresensors, such as a camera sensor or gas sensor. The camera sensor isused to capture images of the flange. The gas sensor is used to detecthazardous or flammable gas plumes. In some embodiments, a mobileapplication executing on the industrial tablet prompts the operator tocapture images at predetermined angles of image capture. Thepredetermined angle of image capture is based on a map of a systemincluding flanges. In some embodiments, the operator is guided to thelocation of a flange via prompts from the mobile application. Inexamples, guidance to the location of the flange is based on a map of asystem including flanges. In examples, guidance to the location of theflange is based on GPS information captured by the industrial tablet.

At block 604, a condition of the flange is classified using a trainedmachine learning model. In some embodiments, the trained machinelearning model is trained as described with respect to FIGS. 3 and 4 .For example, the trained machine learning model is a trainedconvolutional neural network (CNN) that obtains the images of the flangeas input and outputs the classified condition, including a defect. Inanother example, the trained machine learning model includes segmentingthe images of the flange; and image segmentation, and classifying thecondition of the flange using a YOLO network. In some embodiments, themachine learning model is trained using a simulated dataset withsynthetic flange images (e.g., dataset 500 of FIG. 5 ) that mimicreal-world flange images. In some embodiments, the trained machinelearning model is lightweight and deployed to an industrial tablet withlight calculations used to output flange integrity classifications.

At block 606, an indication of the condition of the flange is rendered.In some embodiments, augmented reality elements corresponding to adefect associated with the condition of the flange are superimposed onan image of the flange in real time. The image of the flange is renderedvia a display (e.g., displays 202 of FIG. 2 ) of the industrial tablet.In examples, the augmented reality elements correspond to a correctivemeasure responsive to the condition of the flange, and is superimposedon the image of the flange in real time. In examples, a bolt andcorresponding nut are augmented reality elements superimposed on animage of the flange rendered on a display (e.g., display 202C of FIG. 2), illustrating inserting the bolt into the bolt holes of the flange andsecuring the nut to the bolt. In some embodiments, maintenanceinstructions and bolt tightening data (bolting patterns, torque andtensioning figures, procedures, techniques and recommended controlledbolting equipment) are rendered based on the condition of the flange.This will enable the user to correct the condition (e.g., correct thedefect) at the time of inspection, if feasible.

In this manner, the present techniques enable classifications of flangeintegrity at a lower cost when compared to traditional techniques. Thepresent techniques enable flange integrity detection and correction byan operator with little to no experience and training required.Moreover, the flange integrity classification according to the presenttechniques results in less time spent per inspection when compared totraditional techniques. The images captured for input to the trainedmachine learning model, and the classifications the condition of theflange establishes a complete and accurate record of the health of asystem including multiple flanges. In some embodiments, the health of asystem including multiple flanges is recorded through the detection ofmultiple anomalies or defects. This automated historical data is storedlocally, at the industrial tablet. The automated historical data istransmitted to a cloud location or server when data transmission isavailable.

FIG. 7 is a schematic illustration of an example controller 700 (orcontrol system) for flange integrity classification using artificialintelligence according to the present disclosure. For example, thecontroller 700 may be operable according to the process 600 of FIG. 6 ,using the mobile application and included in an industrial tablet 200 ofFIG. 2 . The controller 700 is intended to include various forms ofdigital computers, such as printed circuit boards (PCB), processors,digital circuitry, or otherwise parts of a system for supply chain alertmanagement. Additionally the system can include portable storage media,such as, Universal Serial Bus (USB) flash drives. For example, the USBflash drives may store operating systems and other applications. The USBflash drives can include input/output components, such as a wirelesstransmitter or USB connector that may be inserted into a USB port ofanother computing device.

The controller 700 includes a processor 710, a memory 720, a storagedevice 730, and an input/output interface 740 communicatively coupledwith input/output devices 760 (for example, displays, keyboards,measurement devices, sensors, valves, pumps). Each of the components710, 720, 730, and 740 are interconnected using a system bus 750. Theprocessor 710 is capable of processing instructions for execution withinthe controller 700. The processor may be designed using any of a numberof architectures. For example, the processor 710 may be a CISC (ComplexInstruction Set Computers) processor, a RISC (Reduced Instruction SetComputer) processor, or a MISC (Minimal Instruction Set Computer)processor.

In one implementation, the processor 710 is a single-threaded processor.In another implementation, the processor 710 is a multi-threadedprocessor. The processor 710 is capable of processing instructionsstored in the memory 720 or on the storage device 730 to displaygraphical information for a user interface on the input/output interface740.

The memory 720 stores information within the controller 700. In oneimplementation, the memory 720 is a computer-readable medium. In oneimplementation, the memory 720 is a volatile memory unit. In anotherimplementation, the memory 720 is a nonvolatile memory unit.

The storage device 730 is capable of providing mass storage for thecontroller 700. In one implementation, the storage device 730 is acomputer-readable medium. In various different implementations, thestorage device 730 may be a floppy disk device, a hard disk device, anoptical disk device, or a tape device.

The input/output interface 740 provides input/output operations for thecontroller 700. In one implementation, the input/output devices 760includes a keyboard and/or pointing device. In another implementation,the input/output devices 760 includes a display unit for displayinggraphical user interfaces.

There can be any number of controllers 700 associated with, or externalto, a computer system containing controller 700, with each controller700 communicating over a network. Further, the terms “client,” “user,”and other appropriate terminology can be used interchangeably, asappropriate, without departing from the scope of the present disclosure.Moreover, the present disclosure contemplates that many users can useone controller 700 and one user can use multiple controllers 700.

Implementations of the subject matter and the functional operationsdescribed in this specification can be implemented in digital electroniccircuitry, in tangibly embodied computer software or firmware, incomputer hardware, including the structures disclosed in thisspecification and their structural equivalents, or in combinations ofone or more of them. Software implementations of the described subjectmatter can be implemented as one or more computer programs. Eachcomputer program can include one or more modules of computer programinstructions encoded on a tangible, non-transitory, computer-readablecomputer-storage medium for execution by, or to control the operationof, data processing apparatus. Alternatively, or additionally, theprogram instructions can be encoded in/on an artificially generatedpropagated signal. The example, the signal can be a machine-generatedelectrical, optical, or electromagnetic signal that is generated toencode information for transmission to suitable receiver apparatus forexecution by a data processing apparatus. The computer-storage mediumcan be a machine-readable storage device, a machine-readable storagesubstrate, a random or serial access memory device, or a combination ofcomputer-storage mediums.

The terms “data processing apparatus,” “computer,” and “electroniccomputer device” (or equivalent as understood by one of ordinary skillin the art) refer to data processing hardware. For example, a dataprocessing apparatus can encompass all kinds of apparatus, devices, andmachines for processing data, including by way of example, aprogrammable processor, a computer, or multiple processors or computers.The apparatus can also include special purpose logic circuitryincluding, for example, a central processing unit (CPU), a fieldprogrammable gate array (FPGA), or an application specific integratedcircuit (ASIC). In some implementations, the data processing apparatusor special purpose logic circuitry (or a combination of the dataprocessing apparatus or special purpose logic circuitry) can behardware- or software-based (or a combination of both hardware- andsoftware-based). The apparatus can optionally include code that createsan execution environment for computer programs, for example, code thatconstitutes processor firmware, a protocol stack, a database managementsystem, an operating system, or a combination of execution environments.The present disclosure contemplates the use of data processingapparatuses with or without conventional operating systems, for example,LINUX, UNIX, WINDOWS, MAC OS, ANDROID, or IOS.

A computer program, which can also be referred to or described as aprogram, software, a software application, a module, a software module,a script, or code, can be written in any form of programming language.Programming languages can include, for example, compiled languages,interpreted languages, declarative languages, or procedural languages.Programs can be deployed in any form, including as stand-alone programs,modules, components, subroutines, or units for use in a computingenvironment. A computer program can, but need not, correspond to a filein a file system. A program can be stored in a portion of a file thatholds other programs or data, for example, one or more scripts stored ina markup language document, in a single file dedicated to the program inquestion, or in multiple coordinated files storing one or more modules,sub programs, or portions of code. A computer program can be deployedfor execution on one computer or on multiple computers that are located,for example, at one site or distributed across multiple sites that areinterconnected by a communication network. While portions of theprograms illustrated in the various figures may be shown as individualmodules that implement the various features and functionality throughvarious objects, methods, or processes, the programs can instead includea number of sub-modules, third-party services, components, andlibraries. Conversely, the features and functionality of variouscomponents can be combined into single components as appropriate.Thresholds used to make computational determinations can be statically,dynamically, or both statically and dynamically determined.

The methods, processes, or logic flows described in this specificationcan be performed by one or more programmable computers executing one ormore computer programs to perform functions by operating on input dataand generating output. The methods, processes, or logic flows can alsobe performed by, and apparatus can also be implemented as, specialpurpose logic circuitry, for example, a CPU, an FPGA, or an ASIC.

Computers suitable for the execution of a computer program can be basedon one or more of general and special purpose microprocessors and otherkinds of CPUs. The elements of a computer are a CPU for performing orexecuting instructions and one or more memory devices for storinginstructions and data. Generally, a CPU can receive instructions anddata from (and write data to) a memory. A computer can also include, orbe operatively coupled to, one or more mass storage devices for storingdata. In some implementations, a computer can receive data from, andtransfer data to, the mass storage devices including, for example,magnetic, magneto optical disks, or optical disks. Moreover, a computercan be embedded in another device, for example, a mobile telephone, apersonal digital assistant (PDA), a mobile audio or video player, a gameconsole, a global positioning system (GPS) receiver, or a portablestorage device such as a universal serial bus (USB) flash drive.

Computer readable media (transitory or non-transitory, as appropriate)suitable for storing computer program instructions and data can includeall forms of permanent/non-permanent and volatile/non-volatile memory,media, and memory devices. Computer readable media can include, forexample, semiconductor memory devices such as random access memory(RAM), read only memory (ROM), phase change memory (PRAM), static randomaccess memory (SRAM), dynamic random access memory (DRAM), erasableprogrammable read-only memory (EPROM), electrically erasableprogrammable read-only memory (EEPROM), and flash memory devices.Computer readable media can also include, for example, magnetic devicessuch as tape, cartridges, cassettes, and internal/removable disks.Computer readable media can also include magneto optical disks andoptical memory devices and technologies including, for example, digitalvideo disc (DVD), CD ROM, DVD+/-R, DVD-RAM, DVD-ROM, HD-DVD, and BLURAY.The memory can store various objects or data, including caches, classes,frameworks, applications, modules, backup data, jobs, web pages, webpage templates, data structures, database tables, repositories, anddynamic information. Types of objects and data stored in memory caninclude parameters, variables, algorithms, instructions, rules,constraints, and references. Additionally, the memory can include logs,policies, security or access data, and reporting files. The processorand the memory can be supplemented by, or incorporated in, specialpurpose logic circuitry.

Implementations of the subject matter described in the presentdisclosure can be implemented on a computer having a display device forproviding interaction with a user, including displaying information to(and receiving input from) the user. Types of display devices caninclude, for example, a cathode ray tube (CRT), a liquid crystal display(LCD), a light-emitting diode (LED), and a plasma monitor. Displaydevices can include a keyboard and pointing devices including, forexample, a mouse, a trackball, or a trackpad. User input can also beprovided to the computer through the use of a touchscreen, such as atablet computer surface with pressure sensitivity or a multi-touchscreen using capacitive or electric sensing. Other kinds of devices canbe used to provide for interaction with a user, including to receiveuser feedback including, for example, sensory feedback including visualfeedback, auditory feedback, or tactile feedback. Input from the usercan be received in the form of acoustic, speech, or tactile input. Inaddition, a computer can interact with a user by sending documents to,and receiving documents from, a device that is used by the user. Forexample, the computer can send web pages to a web browser on a user’sclient device in response to requests received from the web browser.

The term “graphical user interface,” or “GUI,” can be used in thesingular or the plural to describe one or more graphical user interfacesand each of the displays of a particular graphical user interface.Therefore, a GUI can represent any graphical user interface, including,but not limited to, a web browser, a touch screen, or a command lineinterface (CLI) that processes information and efficiently presents theinformation results to the user. In general, a GUI can include aplurality of user interface (UI) elements, some or all associated with aweb browser, such as interactive fields, pull-down lists, and buttons.These and other UI elements can be related to or represent the functionsof the web browser.

Implementations of the subject matter described in this specificationcan be implemented in a computing system that includes a back endcomponent, for example, as a data server, or that includes a middlewarecomponent, for example, an application server. Moreover, the computingsystem can include a front-end component, for example, a client computerhaving one or both of a graphical user interface or a Web browserthrough which a user can interact with the computer. The components ofthe system can be interconnected by any form or medium of wireline orwireless digital data communication (or a combination of datacommunication) in a communication network. Examples of communicationnetworks include a local area network (LAN), a radio access network(RAN), a metropolitan area network (MAN), a wide area network (WAN),Worldwide Interoperability for Microwave Access (WIMAX), a wirelesslocal area network (WLAN) (for example, using 802.11 a/b/g/n or 802.20or a combination of protocols), all or a portion of the Internet, or anyother communication system or systems at one or more locations (or acombination of communication networks). The network can communicatewith, for example, Internet Protocol (IP) packets, frame relay frames,asynchronous transfer mode (ATM) cells, voice, video, data, or acombination of communication types between network addresses.

The computing system can include clients and servers. A client andserver can generally be remote from each other and can typicallyinteract through a communication network. The relationship of client andserver can arise by virtue of computer programs running on therespective computers and having a client-server relationship. Clusterfile systems can be any file system type accessible from multipleservers for read and update. Locking or consistency tracking may not benecessary since the locking of exchange file system can be done atapplication layer. Furthermore, Unicode data files can be different fromnon-Unicode data files.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of what may beclaimed, but rather as descriptions of features that may be specific toparticular implementations. Certain features that are described in thisspecification in the context of separate implementations can also beimplemented, in combination, in a single implementation. Conversely,various features that are described in the context of a singleimplementation can also be implemented in multiple implementations,separately, or in any suitable sub-combination. Moreover, althoughpreviously described features may be described as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can, in some cases, be excised from thecombination, and the claimed combination may be directed to asub-combination or variation of a sub-combination.

Particular implementations of the subject matter have been described.Other implementations, alterations, and permutations of the describedimplementations are within the scope of the following claims as will beapparent to those skilled in the art. While operations are depicted inthe drawings or claims in a particular order, this should not beunderstood as requiring that such operations be performed in theparticular order shown or in sequential order, or that all illustratedoperations be performed (some operations may be considered optional), toachieve desirable results. In certain circumstances, multitasking orparallel processing (or a combination of multitasking and parallelprocessing) may be advantageous and performed as deemed appropriate.

Moreover, the separation or integration of various system modules andcomponents in the previously described implementations should not beunderstood as requiring such separation or integration in allimplementations, and it should be understood that the described programcomponents and systems can generally be integrated together in a singlesoftware product or packaged into multiple software products.

Accordingly, the previously described example implementations do notdefine or constrain the present disclosure. Other changes,substitutions, and alterations are also possible without departing fromthe spirit and scope of the present disclosure.

Furthermore, any claimed implementation is considered to be applicableto at least a computer-implemented method; a non-transitory,computer-readable medium storing computer-readable instructions toperform the computer-implemented method; and a computer systemcomprising a computer memory interoperably coupled with a hardwareprocessor configured to perform the computer-implemented method or theinstructions stored on the non-transitory, computer-readable medium.

Particular embodiments of the subject matter have been described. Otherembodiments are within the scope of the following claims. For example,the actions recited in the claims can be performed in a different orderand still achieve desirable results. As one example, some processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults.

What is claimed is:
 1. A computer-implemented method for flangeintegrity using artificial intelligence, the method comprising:obtaining, with one or more hardware processors, images of a flange,wherein the images are captured at a predetermined angle of imagecapture; classifying, with the one or more hardware processors, acondition of the flange using a trained machine learning model; andrendering, with the one or more hardware processors, an indication ofthe condition of the flange.
 2. The computer implemented method of claim1, wherein an augmented reality element corresponding to a defect of theflange is superimposed on an image of the flange in real time.
 3. Thecomputer implemented method of claim 1, wherein an augmented realityelement corresponding to a corrective measure responsive to a defect ofthe flange is superimposed on an image of the flange in real time. 4.The computer implemented method of claim 1, wherein the trained machinelearning model is a trained convolutional neural network (CNN) thatobtains the images of the flange as input and outputs the classifiedcondition.
 5. The computer implemented method of claim 1, whereinclassifying the condition of the flange using the trained machinelearning model further comprises: segmenting the images of the flange;and classifying the condition of the flange using a you only look once(YOLO) network.
 6. The computer implemented method of claim 1, whereinthe indication of the condition of the flange is a message with a typeof defect.
 7. The computer implemented method of claim 1, furthercomprising rendering instructions to correct the condition.
 8. Thecomputer implemented method of claim 1, further comprising generating analert in response to the condition being a hazardous condition.
 9. Anapparatus comprising a non-transitory, computer readable, storage mediumthat stores instructions that, when executed by at least one processor,cause the at least one processor to perform operations comprising:obtaining images of a flange, wherein the images are captured at apredetermined angle of image capture; classifying a condition of theflange using a trained machine learning model; and rendering anindication of the condition of the flange.
 10. The apparatus of claim 9,wherein an augmented reality element corresponding to a defect of theflange is superimposed on an image of the flange in real time.
 11. Theapparatus of claim 9, wherein an augmented reality element correspondingto a corrective measure responsive to a defect of the flange issuperimposed on an image of the flange in real time.
 12. The apparatusof claim 9, wherein the trained machine learning model is a trainedconvolutional neural network (CNN) that obtains the images of the flangeas input and outputs the classified condition.
 13. The apparatus ofclaim 9, wherein classifying the condition of the flange using thetrained machine learning model further comprises: segmenting the imagesof the flange; and classifying the condition of the flange using a youonly look once (YOLO) network.
 14. The apparatus of claim 9, wherein theindication of the condition of the flange is a message with a type ofdefect.
 15. A system, comprising: one or more memory modules; one ormore hardware processors communicably coupled to the one or more memorymodules, the one or more hardware processors configured to executeinstructions stored on the one or more memory models to performoperations comprising: obtaining images of a flange, wherein the imagesare captured at a predetermined angle of image capture; classifying acondition of the flange using a trained machine learning model, whereina machine learning model is trained to classify the condition of theflange using synthetic images; and rendering an indication of thecondition of the flange.
 16. The system of claim 15, wherein the machinelearning model is trained to classify the condition using syntheticimages at a remote server, and a trained machine learning model isdeployed to an industrial tablet.
 17. The system of claim 16, whereinthe trained machine learning model is quantized to generate alightweight trained machine learning model prior to deploying themachine learning model to the industrial tablet.
 18. The system of claim15, wherein the trained machine learning model is a trainedconvolutional neural network (CNN) that obtains the images of the flangeas input and outputs the classified condition.
 19. The system of claim15, wherein classifying the condition of the flange using the trainedmachine learning model further comprises: segmenting the images of theflange; and classifying the condition of the flange using a you onlylook once (YOLO) network.
 20. The system of claim 15, wherein theindication of the condition of the flange is a message with a type ofdefect.